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Introduction 

This report presents the findings from interviews 
with principals about their schools’ participation 
in the ASSISTments program. The purpose of the 
interviews was to explore the following areas: 

• To what extent schools had formal or informal 
policies or expectations reagrding homework 
practice and completion and specifically for 
seventh-grade mathematics 

• To what extent schools had concerns about 
homework completion rates and specifically 
for seventh-grade mathematics 

• Whether the school had existing practices or 
norms for teachers to discuss student learning 
outcomes, and how often teachers met to 
focus on data 

• The extent to which principals met with 
teachers to discuss data 

• What common benchmark assessments the 
school used for seventh-grade mathematics 

• What other initiatives in the district or 
school could impact curriculum, instruction, 
assessment, or professional development and 
what the current priorities were in the school 

• Factors related to the community such as 
broad socio-economic status, or level of 
support for education, that might affect 
teaching and student achievement in the 
school. 


In addition, the principals were asked questions 
specifically about the early implementation of the 
ASSISTments program in their schools. These 
questions concerned: 

• Principals’ impressions about teachers’ 
reactions to the ASSISTments summer 
training and early implementation efforts 

• Teachers’ level of interest in using 
ASSISTments 

• Teachers’ expectations about using 
ASSISTments 

• Factors that might present challenges for 
implementation 

• Factors that would be important to 
the success of the efficacy study and 
implementation. 
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Methodology 


The goal of the study is to conduct a randomized 
control efficacy trial of ASSISTments, an online 
formative assessment and tutoring platform that 
supports teachers in their use of homework 
to improve math instruction and learning. To 
conduct the study, the project team planned to 
recruit classrooms from 52 public schools serving 
students in the 7th grade, with implementation 
lasting two years per cohort of teachers (in 
treatment condition), with a total of two cohorts. 
Cohort 1 sites began in September 2012; Cohort 
2 began in September 2013. The RCT design 
randomly assigned schools to condition, with half 
serving in either the treatment or control group. 

To measure student learning, the project 
team will use math achievement scores on a 
nationally normed achievement test, TerraNova, 
as the outcome measure. The New England 
Common Assessment (NECAP) will be used for 
covariate scores, obtained prior to intervention. 
SmarterBalanced (being introduced in spring 
2014)will be an additional outcome measure. 

The data analytic strategy will include a three- 
-level hierarchical linear regression model 
(students nested within teachers nested within 
schools), and include both moderator (e.g., 
student characteristics, such as low-baseline 
math achievement, special education, ELL, 
socioeconomic background) and mediator analyses 
(e.g., the link between use of the intervention with 
student homework completion rate). 


As a part of a series of data collection activities, 
principals in all participating schools were 
interviewed in the early fall of the first year of their 
school’s participation. 

A research team from the Center for Research and 
Evaluation (CRE) at University of Maine interviewed 
cohort 1 principals from the treatment and control 
sites in October and November 2012. 1 There were 
nine treatment sites and eight control sites, for a 
total of 17 principals. After these interviews, one 
of the treatment schools with one participating 
teacher withdrew from the study, so cohort 1 
had 16 schools for the remainder of the study. 

Two more interviews were conducted later, one in 
January 2013 and one in June 2013. All interviews 
were conducted in person at the school site except 
for one where the principal returned responses by 
email in January 2013. A team of six researchers 
(three faculty and three graduate students) met 
for training on the protocol and conducted the 
interviews. Cohort 1 interviews were digitally 
recorded and transcribed, producing a total of 
633 minutes of interviews and a transcript of 202 
single-spaced pages. Interviews with the treatment 
site principals were longer than those for control 
site principals, about 45 minutes compared with 
30 minutes, because of the additional questions 
about the ASSISTments implementation effort. On 
average, interviews were 37 minutes long and 12 
typed pages (single-spaced) in length. 


1 The interview protocols for principals in the control group 
and the treatment group are in Appendix A. 
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Cohort 2 interviews took place in October and 
November 2013 with principals from the 14 
treatment sites and in September and October 
2014 with principals from the 14 control sites, for 
a total of 28 principals. One faculty member and 
one staff member from the Center for Research 
and Evaluation conducted the 14 interviews by 
phone in fall 2013, and the same individuals, along 
with an SRI researcher conducted the interviews 
in fall 2014. Interviews were digitally recorded 
and transcribed. The cohort 2 principal interviews 
produced 828 minutes of interviews and 286 
pages of transcripts. Interviews for cohort 2 ranged 
from about 20 minutes to 50 minutes, with longer 
interviews resulting from the protocol for treatment 
sites. On average, cohort 2 interviews were 30 
minutes long and 10 typed pages in length. 

After the first round of interviews in fall 2012, data 
were compiled in tables that briefly described 
and summarized the responses and patterns for 
homework policy, school priorities, data practices, 
and concerns about the study. The preliminary 
results were shared informally and posted on the 
shared Google site with the study team. After the 
second round of interviewing in fall 2013, the data 
were reorganized and compiled to include data for 
each principal’s responses, summaries for each of 
the two cohorts, and an overall summary for both 
cohorts (although without the cohort 2 control 
sites). These data form the basis of the analysis 
reported here. Data from the fall 2014 interviews 
were also incorporated for the final analysis 
presented in this report. 


In addition, we decided to collect any formal 
written policies on homework from all schools in 
the study. In fall 2014, we telephoned the main 
school offices and looked for policies posted on 
school or district websites. This provided a more 
accurate source of information on the question of 
whether formal policies existed. Given that several 
of the principals were new to their schools at the 
time of the interviews, some were not yet familiar 
with the policies. We examined the documents 
and tabulated the relevant information to analyze 
the type of policy (district policy manual or school 
handbook), whether the policy addressed the 
general purpose of homework and/ or suggested 
time (number of minutes) and frequency (number 
of days) for homework assignments, and whether 
the policy specifically mentioned expectations 
for math homework. The interviews were useful 
for understanding the principals’ perceptions 
of informal expectations and practices around 
homework in the school. 
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Findings 

We present the findings by broad topics and point 
out the general trends as well as differences across 
the schools or cohorts. Again, where we talk about 
45 participating schools and principals throughout 
this report, one cohort 1 treatment school withdrew 
from the study after the principal interviews were 
conducted. Thus, the study ended up with 44 
participating schools across the two cohorts. 

Homework Policies and Expectations 

Examination of the formal district and school 
policies revealed that the overwhelming majority 
of schools participating in the study did have 
some formal policy concerning homework. There 
were no real differences between the two cohorts 
or the two conditions (control and treatment) on 
whether they had a formal policy. In cohort 1,14 
of the 17 schools had one and in cohort 2, 27 
of the 28 schools did. Homework policies were 
communicated in district policy manuals and 
student or parent handbooks in roughly the same 
proportion. Some of the four schools without a 
formal homework policy indicated they are moving 
toward standards-based reporting of student 
performance and thus are not grading homework 
in the traditional way. 

When we examined the language of the formal 
policies, we found that the cohort 1 schools were 
more likely to have policies that addressed the 
general purpose of homework, while the cohort 2 
schools’ policies were more likely to cover both the 
purpose of homework and the suggested number 


of minutes per night and occasionally frequency of 
homework over the week. In cohort 1, eight schools 
had policies that addressed purpose only, while 
six schools had policies that addressed purpose 
and time on homework. In cohort 2, nine schools 
had policies that addressed purpose only, while 
18 schools’ policies addressed purpose and time. 
Treatment schools were somewhat more likely than 
control schools to have policies suggesting time 
duration of homework. Across the 41 schools with 
some formal homework policy, 17 addressed the 
purpose of homework only while 24 addressed both 
the purpose and suggested time. 

Of the 41 schools with a policy on homework, only 
one school, a cohort 1 control site, mentioned 
expectations specifically for mathematics 
homework. This included a suggestion for the 
number of minutes per night and frequency during 
the week. For all other schools, the policy language 
was very general and covered all content areas 
together. 

While the formal policy documents express the 
intended expectations about homework, the 
interviews helped capture informal expectations 
and enacted homework practice within schools. 
Most of the principals we interviewed were able to 
answer questions about homework expectations 
readily, but a few had been newly appointed at the 
time of the interviews and were not yet familiar with 
the formal policies and informal practices regarding 
homework in their new schools for particular grade 
levels or content areas. They were more hesitant 
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and provided very general responses to the 
questions. Therefore, some caution is needed in 
reviewing the interview data. 

When asked about homework expectations specifically 
for seventh-grade or middle-level mathematics, all 
45 principals indicated that some homework was 
expected for mathematics but that it generally was 
up to the teacher to decide when and how much to 
assign. When asked whether math homework was 
usually graded or counted toward a grade, the majority 
of principals indicated that homework was graded 
for mathematics (38 of 45). Four principals indicated 
that homework was not graded, and three principals 
said that it was sometimes graded. Again, some of 
these schools were in transition to a standards-based 
proficiency model. 

Frequency and Duration of Homework 

Of the 41 schools that had a formal policy on 
homework, 24 provided guidelines on the number of 
minutes per night or frequency of homework for all 
subjects. Most of the guidelines for middle grades 
or seventh-grade specifically were in the 60- to 
90-minute range per night. Several schools used 
the 10-minute per grade level guideline, suggesting 
70 minutes per night for seventh-grade homework 
across all subjects. Frequency for homework was 
mentioned less often in the policies. Four schools 
had policies that suggest homework be assigned 
four nights per week. Others indicated homework 
should be assigned “nightly” or did not specify a 
frequency. Again, only one school had a policy that 
specifically mentioned homework expectations for 
math: 30 minutes per night and four nights a week. 


In the interviews, principals consistently said that 
homework in general (for various subjects) was 
typically assigned four nights a week and typically 
not on Friday night because of sports and extra¬ 
curricular events. No principal reported homework 
being assigned less than three nights per week. 

A common expectation for seventh graders was 
that all homework combined should take about 
70 minutes per night. Many principals explained 
that this was a district-level standard of 10 minutes 
per grade level. Some principals gave a range in 
time; 45 minutes was frequently the minimum time 
that students should be working on homework in 
general across various subjects, and no principal 
reported that students should be given more than 
2 hours of homework nightly. 

When asked about the frequency and amount of 
homework expected for mathematics specifically, 
principals also consistently reported that math 
homework was typically assigned four nights per 
week and in no cases should be given less than 
twice weekly. Ten to 30 minutes per night was cited 
as the minimum expectation for the duration of math 
homework, and 60 minutes was the longest time 
that students would be expected to work on math 
homework each night. When the two conditions 
were compared, no control school principal said 
that math homework should take longer than 30 
minutes per night. Note that for treatment schools, 
the principals more often mentioned that decisions 
about homework frequency were up to the math 
teachers or teaching team. When we examined the 
expectations for time spent on homework for all 
homework and for math homework specifically, it 
was apparent that schools/ principals expected 
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students to spend a larger portion of their 
homework time on math than on other subjects. 
This expectation is consistent with the goal of 
many of the participating schools to improve 
students’ performance on math assessments cited 
by 18 of the 45 principals we interviewed. 

Concerns About Homework Completion 

Homework completion was a concern for many 
of the principals, with almost a third (15) indicating 
that it was a definite concern for their students 
in general. Another 19 principals stated that 
homework completion was a concern for a subset 
of their students, estimated to be between 5% 
and 30% of students. Similar concerns were 
voiced about homework completion for seventh- 
grade mathematics specifically. The consistent 
reason given for noncompletion of homework 
was the lack of parental support for homework 
in some students’ homes. Other principals cited 
lack of student effort or motivation. Two cohort 
2 treatment principals and four cohort 2 control 
principals noted that homework completion was 
not a concern at all. Principals described efforts 
their schools had implemented to provide more 
homework support and tutoring to students before 
school, after school, or during the school day. 

Some representative comments from principals are 
provided below. A comprehensive list of quotations 
is provided in Appendix B. Principals voicing 
minimal or modest concern about homework 
completion shared the following comments: 

Do we have a small percentage of kids who are 

negligent or let’s just say lazy, unmotivated to do 


it? In some cases the environment at home is 
not conducive, and we understand that as well. 

We have not had a significant problem with 
kids getting their math homework in. Both of 
our middle school teachers do not overburden 
kids with so-called “busy work.” Also, they 
allow for class time to work with those few who 
do not get it in on time, or these same kids do 
attend our afternoon study sessions. 

Yes, and it’s based upon motivation, based 
upon home support. So I don’t have a number 
about it but depending upon the age group 
especially. 

Principals voicing higher levels of concern about 
homework completion said: 

Yes, that’s one of our biggest problems. We 
feel it is lack of motivation for kids to do - to 
complete work, to complete it in a timely 
fashion and to put enough energy and effort 
into it to make it worthwhile.... When a child 
is at the level where they’re missing three 
assignments they stay after school every night, 
five days a week until they’re caught up. 

I would say that I hear it more from the math 
teacher. I’m not sure it’s math specific in terms 
of completion. I haven’t heard so much in 
seventh-grade EU\ or seventh-grade social 
studies but definitely hear it from seventh-grade 
math. 

Yeah, I think that they think that homework 
completion is a problem, and I think specifically 
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within math, the inability of students to 
complete, um, homework, it just is a problem. 

Yes. And, you know, and what we’ve just 
implemented this year using some resources, 
our Title I resources, to offer an opportunity for 
students to have an hour of help, if you will, 
with their homework. And it seems that a lot of 
our students are seeking that help for math. 

Principals’ concerns about homework completion 
were consistent with what we heard in the teacher 
interviews 2 , in which a majority of teachers 
expressed some concern about homework 
completion, often for a significant portion of their 
students (more than 25-30%). 

Data Use Practices 

We asked a series of questions to understand 
the extent to which and how teachers at the 
participating schools were collaborating at the start 
of the study on examining and using student data 
for decision making, what expectations principals 
held, and principals’ involvement in this effort. 

When asked if teachers met in grade-level teams, 
content teams, or both, principals indicated that 
their schools were just as likely to have teachers 
meet in grade-level teams as in content teams. 
Twenty-eight principals said they have grade-level 
teams, and 29 said they had content teams. Small 
grade schools were more likely to meet in grade- 
level teams only, while at middle schools teachers 
often met in both grade-level teams and content 

2 Findings from teacher interviews will be analyzed and shared 
in a separate report. 


teams. There were no significant differences by 
cohort or by condition. 

In terms of frequency of teacher team meetings, 
principals indicated that teachers meet more 
frequently in grade-level than in content teams. 
About a third (17) of the principals said that their 
teachers met in grade-level groups at least weekly 
(with six meeting daily), and 19 principals said their 
content teams met at least monthly (eight of which 
met weekly). Another five principals said that their 
curricular teams met only quarterly. 

While there was not a large difference between 
control and treatment schools in these frequencies, 
it is noteworthy that of the treatment schools, 
more of the cohort 2 schools (10 of 14) indicated 
having weekly grade-level teacher meetings than 
cohort 1 schools (3 of 9). It would be interesting 
to survey teachers near the end of their two-year 
participation to see whether any changes occurred 
in how they meet to examine and discuss student 
performance data and whether the ASSISTments 
intervention triggered more change in treatment 
schools than control schools. 

When asked about their own participation in these 
team meetings, 18 of the 45 principals said they 
regularly attended the meetings monthly or more 
frequently, while 16 indicated that they minimally 
interacted with teachers in this capacity. Half the 
principals who indicated monthly (or more frequent) 
attendance were from the cohort 2 control 
condition, which was sampled most recently, 
so this may represent a unique population or a 
changes in the expectations of administrators 
regarding team meetings. Overall, however (and 
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more specifically for treatment schools), the 
responses indicated a low level of engagement 
by principals directly in teacher team meetings on 
student performance data. 

The interviews probed principals’ expectations for 
how teachers should spend their team meeting 
time and expectations about teachers examining 
student data more generally. More than half (26) of 
the principals explicitly stated they wanted teachers 
to have discussions during team meetings based 
on student performance data. Although cohort 2 
treatment schools had more frequent grade-level 
teacher meetings, cohort 1 treatment principals 
more frequently stated that those meetings should 
include data-driven discussion (8 of 9) than their 
cohort 1 control counterparts (5 of 8), the cohort 
2 control principals (9 of 14), or the cohort 2 
treatment principals (4 of 14). Only 10 principals 
offered more specifics about what they wanted 
teachers to discuss. The topics included improving 
assessment scores, focusing on student learning 
needs, being proactive, planning for interventions, 
developing more hands-on approaches, and 
using ASSISTments more. Some representative 
statements about expectations for teachers to 
examine data included the following from treatment 
site principals from both cohorts: 

They’re expected to look at the data. We’ve 
actually generated these huge—we started 
them—data sheets on each class last year 
that lists the NWEA (Northwest Evaluation 
Association) scores, their NECAP (New 
England Common Assessment Program) 
scores, and their writing prompt scores, 


and we’ve color coded them for ones that 
are receiving RTI (Response to Intervention) 
services, ones that receive special ed services. 
And then we’ve looked at the kids who are 
falling below the “meets” [proficiency] level and 
trying to figure out how to address their needs. 

We want them to discuss [in monthly content 
team meetings] how students are doing. We 
want to know overall, like what percentage of 
them are showing growth, what percentage is 
not. What percentage of students is in each 
category. And then the students who are the 
lowest, what are we doing to bring them up? 

Constantly. So we have one meeting a month, 
one meeting a week. It’s called Common 
Planning Time, and they are supposed to 
spend that time on curriculum and assessment. 
Which is hardly enough time. And then 
Wednesday’s are the vertical and the team. 

I would assume they are discussing that every 
day.... I walk in on teachers all the time. They 
are sitting down. They are saying, “I had so 
and so last year. You have him this year. You 
know I am having trouble with this or that. Can 
you help?” We have RTI meetings on kids and 
develop plans together. 

And that’s really what, you know, what we’re 
asking teachers to examine. So they’re looking 
at student work, they’re looking at the results of 
student work, they’re looking at a breakdown 
of the results of those students, student works, 
to determine where there may be shortcomings 
in either our curriculum or in our instruction. 
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They should be living it and dreaming it. We’re 
very conscious of our state assessment tests, 
not that we are, um, ... driven by them, but we 
are certainly guided by the expectations. We do 
want our students to do well. We do a lot of 
work in looking at the assessment and looking 
at how our students perform from the previous 
year. And then we focus our instruction on the 
areas that students need to improve on for the 
following year. So that, you know, that end- 
of-the-day time, especially when we get back 
our data, the staff work on that together. And 
usually it’s in content groups. 

That should be driving their instruction, you 
know, we look at them in terms, we do RTI 
regularly and benchmark assessments, so that 
we know what direction to go in with them 
individually and as a group. 

Whereas a majority of principals did express 
the expectation that teachers would examine 
student data during their team meetings, others 
said that teachers engaged in data discussion 
during their professional learning community 
(PLC) time or inservice days. Notably, four 
cohort 2 control principals indicated a focus 
on examining data for the RTI process 
and for identifying which students needed 
interventions. 

We found a disconnect between principals’ stated 
goals and actions regarding data use in their 
schools. Principals were fairly emphatic about their 
expectation that teachers would examine student 
data to inform instruction, yet they indicated 
minimal direct involvement as principals in that 


effort. It would be interesting to survey principals 
at the conclusion of their two-year participation 
in the ASSISTments project to see whether they 
report any changes in their own expectations for 
teachers’ use of data, leadership practice, and 
involvement with teachers regarding data use, and 
whether the ASSISTments intervention triggered 
more change in this regard in treatment schools 
than in control schools. 

Use of Common Benchmark Assessments 

The NWEA Measures of Academic Progress® 

(MAP) test and NECAP test were the most 
commonly used assessments by schools to 
provide data about students for the teacher 
meeting discussions. Of the 45 schools, the NWEA 
and NECAP assessments were used in 34 schools 
and 20 schools, respectively. At the time of the 
interviews, cohort 1 control schools used these two 
assessments more commonly than the treatment 
schools. All control schools used the NWEA, and 
6 out of 8 used the NECAP. For cohort 2, more 
treatment schools reported using the NECAP (9 of 
14 versus 2 of 14), while far more control schools 
reported administering the NWEA (13 or 14 versus 
8 of 14). Principals from three cohort 2 treatment 
schools described using universal screening for 
their RTI programs, and another several schools 
in all cohorts were using curriculum-based 
or common assessments. The most recently 
interviewed principals, cohort 2 control, were 
the only ones (8 of 14) that indicated giving the 
Smarter Balanced test, which is aligned with the 
Common Core standards. Other assessments that 
principals mentioned were TerraNova (3 schools), 
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aimsweb tool (3), Northwestern (1), Scholastic 
Math Inventory (1), STAR Assessments (1), and 
the Iowa Algebra test (1). One principal indicated 
that the school used no formal assessments, and 
another principal indicated the school is in a state 
of flux and planned to change assessments. 

Other Initiatives and Priorities for 
Participating Schools 

Across the 45 principals, we found that schools 
focused on a myriad of school and district 
initiatives. About half (21) of the principals 
mentioned a shift to implement the Common Core 
standards, with three of these principals mentioning 
specific software and curriculum packages used 
to that end. It is noteworthy that while half the 
principals discussed this as a major priority, only 
two cohort 1 control schools mentioned it as a 
school focus. 

The implementation of RTI was the second most 
commonly cited initiative, being a major focus of 
17 schools. Principals of these schools stated that 
this paradigm shift was a major priority, with five 
of the cohort 2 treatment schools adding that this 
initiative included movement toward a competency- 
or proficiency-based high school diploma, and one 
cohort 2 control school specifically connecting the 
RTI program to math achievement. Treatment site 
principals described this effort in their schools: 

And the other thing that we’re working very 
hard on is our RTI process. We’re fortunate 
that we have early release on Fridays that 
we devote to PLCs. So that is when we look 


at students and monitor their process and 
evaluate data. 

So currently they’re working on, looking at the 
architecture of how we have RTI throughout 
our school ... supporting students who are 
not meeting standards so, you know, they 
generate that goal at the beginning of the year. 
Sometimes it’s based around the math goal or 
the literacy goal or the vocab goal, and then 
they work on it and review their data every 
other week and plan from that. 

About a quarter (12) of the principals said that 
literacy/reading/language arts was a focus, with no 
major difference in frequency between cohort or 
condition. 

Other initiatives were mentioned by fewer 
principals. Six discussed their efforts at Positive 
Behavior Intervention Support. Cohort 2 schools 
exclusively mentioned the implementation of 
teacher evaluation systems as a priority (14) and 
discussed programs geared toward individualized 
or customized learning (9). This finding may have 
been influenced by the timing of the principal 
interviews as all the cohort 2 interviews occurred in 
fall 2013 or later, at least one year after the cohort 
1 interviews, and in the interim the state made 
teacher evaluation a priority. 

Less frequently named priorities were the switch 
to and specific training for iPads (4), antibullying 
programs (3), being a Continuing Improvement 
Priority School (3), implementing science initiatives 
(2), increasing the use of data in pedagogical 
practice (2), having an inquiry-based approach 


Principals Discuss Early Implementation of the ASSISTments Online Homework Tutor for Mathematics 
© Copyright 2015 SRI International. 


10 


SRI Education 


as an International Baccalaureate school (1), 
using open-ended questions for deeper student 
engagement (1), professional development on 
math instruction (1), suicide counseling (1), the 
enhancement of 21st century skills (1), and 
combining 7th- and 8th-grade classrooms (1). 

Teacher and Principal Expectations 
About ASSISTments 

Principals from treatment schools (9 in cohort 1 
and 14 in cohort 2) were asked how interested their 
teachers were in using ASSISTments and what 
expectations the teachers and they themselves 
had about the use of ASSISTments in their school. 
Broadly, principals consistently indicated that 
their teachers were very interested in using this 
homework tool, that they had expressed positive 
views of the summer training, and were eager to 
begin or had already begun to use ASSISTments 
in their classrooms at the time of the principal 
interviews. Principals had heard only about minor 
aspects that were frustrating initially to teachers, 
related to the normal process of learning to use a 
tool for the first time and convincing students to 
enter their homework answers into the program 
after they had solved them on paper. 

Comments revealed the expectation that 
ASSISTments would help teachers use their class 
time more efficiently, provide a quick window 
into individual students’ understanding to inform 
teaching and differentiation, provide a broader 
range of information about student learning 
and homework effort, and give students instant 
feedback on their homework answers. Some 


representative views from treatment site principals 
in both cohorts included the following: 

I think the big thing is the time savings that they’ll 
get... and that’s what interested me a lot about 
it, was they have a pretty clear picture of where 
kids are solid on the concepts or the skills or 
where they really need to do some reteaching or 
reinforcing before the kids even walk in the door 
that day, which is amazing.... If the homework 
is corrected before you even get there then you 
jump right into the reinforcing. And I think that 
is going to be a great time saver. So that’s their 
expectation, is to help them with that part, that’s 
the biggest.... It’s really going to bring to light 
the kids that are really struggling I think, who 
are maybe just the outliers. And that becomes 
an individual student conversation. So I think it’s 
going to be helpful there. And then it gives them 
opportunity to work with small groups and really 
plan out how they’re going to differentiate. 

I think it allows him to track students’ progress 
better. Better, easier, quicker, once he, you 
know, goes through that learning process. So, 
the tracking. He thinks, in the conversations 
that I’ve had with him, that kids are getting it. 

They’re getting far more information about 
students, about all of their students than they 
were ever able to devise on their own. And so, 
you know, they can see, you know, and it’s not 

always the same students who struggle. 

So that I think that they are excited about what 
they’re seeing in the in the math portion of their 
kid’s brain. 
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Very interested. They are very interested in 
making it work. They’ve spoken positively 
about it. ...They are hoping that it’s going to 
help them be more effective with using their 
classroom time. I think that they see it as 
providing some useful formative information 
and helping identify areas of concern for 
particular students. You know, being able to 
be a little bit more targeted. 

One principal described how a veteran teacher 
was thinking about retiring but stayed and was very 
enthusiastic about having this homework tool: 

She, you know, was contemplating retiring 
this year coming, which would have been a 
real loss for our school and for our kids. And 
she was skeptical because she has seen 
everything. You know how veteran teachers 
are. But she said, I spoke with her yesterday, 
and she said, “[Name], this is the best thing! I 
wish I would have had this in my first year of 
teaching.” And what she stressed most with 
her kids was the instant feedback for kids. 

Concerns Specific to the 
ASSISTments Study 

Principals from treatment schools (23) were also 
asked to share any concerns about implementing 
ASSISTments in their schools and what factors would 
be critical for the success of the study. Hardware 
issues and limited access to the Internet for students 
living in rural areas were among principals’ more 
common concerns. One principal said, 


...for kids who don’t have computer access 
at home, which are quite a few. So we had 
to accommodate those kids to be able to be 
on the same page and part of the work, which 
I believe—I can’t remember exactly how he 
solved that, but I’m pretty sure kids come in in 
the morning to do their work. 

Another principal indicated not anticipating needing 
LCD projectors to implement ASSISTments: 

The biggest barrier was, How are we going 
to make sure that each classroom had the 
technology? I mean, we didn’t have extra LCD 
projectors hanging around. So I have had to tie 
them up, you know, because they need them 
in their classrooms. It would have been helpful I 
think if people know ahead of time. 

Three cohort 1 principals expressed concern 
about students not being able to take their laptops 
home from school because of either school policy 
or noncompliance from parents. Two cohort 2 
principals mentioned that they had heard of issues 
using ASSISTments with iPads at an early point 
in the school’s implementation of the devices 
that year. Issues related to using iPads with 
ASSISTments were communicated to the team 
and were resolved. 

Six principals were concerned about whether 
teachers would have sufficient time and focus 
to incorporate ASSISTments into their practice. 

This concern was primarily found among cohort 2 
principals (one from cohort 1, five from cohort 2). 
Three cohort 1 principals were concerned with the 
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delayed launch of using ASSISTments at the start 
of the school year because of the administration of 
testing in September. 

Software functionality was a concern for four 
principals. Three cohort 2 principals noted that 
early and ongoing support from Worcester 
Polytechnic Institute (WPI) was a crucial concern, 
two cohort 1 principals were apprehensive about 
whether there was sufficient flexibility within 
the program for teachers to make any needed 
adjustments, one cohort 2 principal lamented 
the absence of prebuilt hints, and one mentioned 
the missing ability to require units with numerical 
answers. One principal also noted that students 
found the red “X” response to errors discouraging. 

When asked what factors would contribute to the 
success of the study, the most commonly voiced 
factor was teacher buy-in. Five principals were 
concerned about this (two in cohort 1, three in 
cohort 2), and two of them noted that having such 
buy-in was most likely contingent on teachers 
seeing results with their students. Related to that 
idea, one principal from each cohort stated that 
it was imperative that the school follow through 
with implementation long enough to see an impact 
(through higher test scores). 

Another concern shared by four principals (two 
from each cohort) was that technical roadblocks 
would have to be minimal (e.g., log-in issues, data- 
entry errors in answers) for the study to succeed. 

The availability of support and continued training 
for teachers was mentioned by four principals (two 
in cohort 1, two in cohort 2). Additional concerns 


included having fidelity of implementation across 
teachers and schools (1), gaining support of 
parents (1), and the need of teachers to maintain 
focus throughout the study (1). 

Summary 

Analysis of the formal school or district policies on 
homework revealed that the vast majority (41 of 45) 
of the schools in cohorts 1 and 2 did have a formal 
policy articulating that homework was generally 
expected. Only one school had a policy that 
specifically mentioned homework expectations for 
math. Of the 41 schools with policies, 24 included 
guidelines on the number of minutes per night 
and days per week for homework assignments. 
Cohort 1 school policies were more general and 
described the purpose of homework, while cohort 
2 policies included both general purpose and 
suggested time and frequency for homework. More 
of the treatment schools included suggestions for 
time and frequency than the control schools. For 
policies that suggested time for homework, the 
typical guideline was 60-90 minutes per night for all 
grades, and several policies suggested 70 minutes 
specifically for seventh-grade homework across all 
subjects. Most of the policies indicated homework 
was expected daily or did not specify frequency. 

In the interviews, all 45 principals indicated that 
some homework was expected for math, but that 
it was generally up to the teacher to decide when 
and how much to assign. The majority of principals 
(38 of 45) indicated that math homework was 
generally graded. Some schools were in transition 
to standards-based proficiency grading, and were 
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reexamining their practice for grading homework or 
counted homework more toward effort than part of 
academic performance. 

Consistently principals said that homework for 
math specifically was typically assigned four nights 
per week, Monday through Thursday. Principals 
indicated a minimum expectation of 10-30 minutes 
per night for math homework and 60 minutes as 
the maximum. Almost half the principals said their 
schools had concerns about students not completing 
homework, and this was especially true for math. This 
concern was echoed in the teacher interviews. 

A majority of schools used grade-level teams for 
teachers to plan and review student data and 
content-specific teams to work on curriculum. 
Principals indicated that teachers met more 
frequently in grade-level teams than in content 
teams. Those with grade-level teams met weekly, 
while content teams met monthly. However, about 
one-quarter of the schools with teacher teams (all 
from cohort 2) met far more regularly, with six grade- 
level teams meeting daily and eight curricular teams 
meeting weekly. Five principals said their content 
teams met even less often, on a quarterly basis. 

While principals indicated an expectation for 
teachers to use their team meetings to discuss 
student performance data, only 18 of the 31 
principals said that they regularly attended these 
meetings monthly or more frequently. 

The NWEA MAP and NECAP were the most 
common benchmark assessments used in their 
schools. Of those schools interviewed most 
recently, the Smarter Balanced test has apparently 


taken the place of the NECAP. Other tests were 
mentioned by three or fewer schools. 

Principals listed a variety of initiatives and 
priorities under way in their schools beyond their 
involvement in ASSISTments. About half the 
principals mentioned implementing the Common 
Core standards, which affected the instructional 
materials teachers were asked to use for 
mathematics in 2013-14. Response to Intervention 
was mentioned by 17 principals as a high priority, 

12 indicated a priority in literacy/ reading/language 
arts, and 18 specifically mentioned that math was 
a priority. Some schools were also shifting away 
from the use of laptops and implementing iPads for 
students in fall 2013. 

Principals said their teachers were quite positive 
about the ASSISTments tool and its potential for 
increasing efficiency in homework review and 
providing more in-depth understanding of students’ 
learning and homework effort through the instant 
feedback to students on homework. 

Broadly, principals did not indicate serious concerns 
about their school’s participation in the study. Six 
principals did indicate some concern about whether 
teachers would have time to integrate ASSISTments 
into their practice given the other initiatives taking 
place in their school. Five principals indicated that 
technical training and support would be critical for 
the success of the study. 

A few principals (three or four) mentioned concerns 
about potential technology and logistics issues. 

One concern for the small rural schools was the 
issue of Internet connectivity in the homes, and 
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whether all students in a seventh-grade math class 
would be able to take their laptops home at night. 
Some of these classes may include advanced 6th- 
graders, and the school may have a policy not to 
allow 6th-graders to take their laptops home. Other 
students’ parents may not have signed the forms 
necessary to allow laptops to go home. 

Concluding Thoughts 

Many of the school principals indicated concerns 
about the high percentage of students who 
consistently did not complete homework. In the 
research literature, low academic expectations have 
been associated with low student achievement. 
Given that these schools joined the ASSISTments 
study largely because of their desire to improve 
student math achievement, it would be interesting to 
see whether their participation in the study motivates 
them to reassess their homework policies or, 
expectations or to institute other actions or supports 
to encourage homework completion, particularly at 
the middle grade level. 

The interviews also indicated that principals 
did not routinely participate in teacher grade- 
level team or content team meetings to discuss 
student performance data. While principals 
said they expected teachers to use this time to 
look at student data, among other things, their 
lack of involvement is a missed opportunity to 
communicate expectations to teachers, to guide 
productive discussions on data, and to hold 
teachers accountable for using data to inform 
curriculum and instruction decisions. This is not 


an uncommon phenomenon in schools today. In 
Maine, the streamlined structure of district central 
offices and persistent state education funding cuts 
mean that school principals often have a broader 
range of job responsibilities than in the past, and 
they do not typically have an assistant principal to 
share these duties. It would be interesting to see if 
principals alter their leadership behavior regarding 
teacher data discussions in part as a result of the 
ASSISTments intervention. 

The participating schools were simultaneously 
engaged in implementing many other broad 
initiatives at the time of the study. Most important, 
they were shifting from the Maine Learning Results 
and state assessment system to the Common 
Core standards (in fall 2013) and Smarter Balanced 
Assessments (in 2013-14). This required teachers 
to learn about the curriculum changes and to 
begin to incorporate instructional materials to 
bring their curriculum up to the higher academic 
expectations for seventh-grade math. Our teacher 
interviews generally indicated that schools were 
holding off on purchasing new textbooks to align 
with the Common Core and were using a mix of 
different instructional resources for the time being. 

In addition, many schools had been focused on 
implementing Response to Intervention strategies 
and improving their literacy or reading achievement. 
Still, despite these demanding times for teachers 
and administrators, neither the principal nor 
teacher interviews indicated strong concerns about 
their ability to complete their participation in the 
study and, for treatment schools, to implement 
ASSISTments in the classroom. 
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There are a number of interesting research 
questions about the potential for the ASSISTments 
intervention to trigger changes in principal and 
or teacher behaviors and school policy and 
expectations for homework. However, these 
changes will remain undocumented unless a 
final exit interview with principals is conducted 
at the conclusion of their participation. For the 
treatment schools, this could be done in spring or 
early fall semester after their participation ends. 

For the control schools, principal exit interviews 
would have to be delayed until after teachers 
participated in the summer training and returned 
to school to begin implementing ASSISTments in 
the classroom. Therefore, it might be possible to 
interview a subsample of principals, for example 
only the treatment school principals. 
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Appendix A: Protocols for Interviews 


Protocols for Interviews: 

• Principal Interview Protocol: Control Group 

• Principal Interview Protocol: Treatment Group 

Principal Interview Protocol: 

Control Group 

Overview of the project 

Thank you again for participating in this important 
research study funded by the U.S. Department 
of Education. This study aims at helping all Maine 
teachers to understand how to use homework 
more effectively to improve student learning. We 
would like to learn from you about district and 
school policies on homework and whether there 
might be any implementation issues. 

Homework Policy 

Let’s start with the school policy. 

1. Do you currently have any general policy on 
homework for 7th grade? 

Is the homework for 7th grade in your school, 

• Required or optional? 

• Graded? 

• Does homework count toward students’ 
course grades? 

Does your district or your school have any 
suggestions or expectations on homework for 
7th graders across all subjects? 


• How many days per week should homework 
be assigned? 

• How long each assignment should be in 
terms of # of problems, and # of minutes? 

• Does your school or district offer any 
guidelines for teachers or parents about how 
much time students are expected to spend 
each night on homework in general? 

2. Do you currently have any general policy on 
homework for Math? 

Is the homework for Math in your school, 

• Required or optional? 

• Graded? 

• Does homework count toward students’ 
course grades? 

Does your district or your school have any 
suggestions or expectations on homework for 
Math? 

• How many days per week should homework 
be assigned? 

• How long each assignment should be in 
terms of # of problems, and # of minutes? 

3. Do teachers in your school feel that homework 

completion is a problem in general? How 

about for math homework in 7th grade? 

4. Do you have any concern with your 
students’ performance in math? 
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Other things going on 

5. What other initiatives related to curriculum, 
assessment, or professional development 
are taking place in your school or district 
currently? 

In what ways might these other initiatives 
impact homework practices or math 
instruction? 

6. What are the major priorities or areas of 
focus currently in your school or district? 

Professional Development Community: 

7. Do teachers at your school meet either in 
content teams or grade level teams each 
week? 

How much time do teachers get to meet in 
their teams each week? 

What are your expectations about how this 
meeting time should be used? 

8. To what extent do you expect teachers to 
discuss student learning outcomes? 

(Probe if teachers are expected to look at 
individual, classroom, or grade level outcomes, 
and if they are expected to use data to make 
decisions about instruction and interventions.) 

9. Do teachers administer common 
benchmark assessments for 7th grade 
math? 


(Which assessments? For what purposes? What 
is done with that data? By school administrators? 
By teachers?) 

10. How often do you meet with math 
teachers to discuss student learning 
data and to strategize about modifying 
instruction, curriculum, or programs? 

School community 

11. Please tell me a little bit about your 
community. 

[a general description of the community, such as 
broad socio-economic factors that may affect 
student learning outcomes. For example, if the 
community is growing or declining, if businesses 
have closed recently resulting in increased 
unemployment, etc.] 

12. How would you describe the students 
attending your school? 

[e.g.,situations such as having kids from different 
countries who speak different languages, having 
high poverty families, etc.] 

13. How would you assess the level of support 
for education in your community? 
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Principal Interview Protocol: 

Treatment Group 

Overview of the project 

[Note: The project debrief can be omitted if 
interview is in person and interviewer has gone 
through the informed consent with the principal. ] 

Thank you again for participating in this important 
research study funded by the U.S. Department 
of Education. This study aims at helping all Maine 
teachers to understand how to use homework 
more effectively to improve student learning. During 
this school year, 7th grade mathematics teachers 
from your school will start assigning homework 
within ASSISTments. Students are expected to 
complete homework on their laptops. We would 
like to learn from you about district and school 
policies on homework and whether there might be 
any implementation issues. 

Homework Policy 

Let’s start with the school policy. 

1. Do you currently have any general policy on 
homework for 7th grade? 

Is the homework for 7th grade in your school, 

• Required or optional? 

• Graded? 

• Does homework count toward students’ 
course grades? 


Does your district or your school have any 
suggestions or expectations on homework for 
7th graders across all subjects? 

[Probe: 

• How many days per week should homework 
be assigned? 

• How long each assignment should be in 
terms of # of problems, and # of minutes? 

• Does your school or district offer any 
guidelines for teachers or parents about how 
much time students are expected to spend 
each night on homework in general? } 

2. Do you currently have any general policy on 

homework for Math? 

Is the homework for Math in your school 

• Required or optional? 

• Graded? 

• Does homework count toward students’ 
course grades 

Does your district or your school have any 
suggestions or expectations on homework for 
Math? 

[Probe for things such as 

• How many days per week should homework 
be assigned? 

• How long each assignment should be in 
terms of # of problems, and # of minutes?] 
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3. Do teachers in your school feel that 
homework completion is a problem in 
general? How about for math homework in 
7th grade? 

4. Do you have any concern with your 
students’ performance in math? 

Implementation 

Worcester Polytechnic Institute (WPI), the provider 
of the ASSISTments technology, hosted the first 
summer professional development workshop 
for participating teachers at Falmouth Middle 
School in August. During the workshop, teachers 
were introduced to features of ASSISTments 
and trained on how to use ASSISTments for 
homework assignments and reviews. The 
workshop discussed how to use ASSISTments to 
make decisions about instruction, to differentiate 
instruction for students, and to use homework 
more effectively. The workshop went well and the 
feedback we have received was largely positive. 

5. Have you heard anything from your 
teachers about the workshop or about the 
study as a whole? 

6. From your perspective, how interested are 
the teachers in using ASSISTments? 

7. What expectations do teachers have related 
to their use of ASSISTments? 

8. We would like to get your perception about 
things that might be difficult for the study. 
Do you foresee any obstacles for either 


students or teachers using ASSISTments 
for homework following our specified 
model that anticipates teachers to use 
ASSISTments for homework assignments 
and reviews 3 times per week and about 20 
minutes for each assignment. What could 
be the barriers? 

9. Do you have any thoughts on whether the 
parents will be supportive of this study? If 
not, why? 

10. Overall, what do you think would be the 
most important factor to the success of 
the study? 

11. How is the implementation of ASSISTments 
going so far in your school? Are there any 
difficulties to date with the implementation? 
(please explain) 

Other things going on 

12. What other initiatives related to curriculum, 
assessment, or professional development are 
taking place in your school or district currently? 

In what ways might these other initiatives 
impact the implementation of ASSISTments, 
homework practices, or math instruction? 

[Schools might have many different things going 
on. Probe for things that might take much time 
from professional development, ability to provide 
release time for teachers, etc.] 

13. What are the major priorities or areas of 
focus currently in your school or district? 
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Professional Development Community: 

14. Do teachers at your school meet either in 
content teams or grade-level teams each 
week? 

How much time do teachers get to meet in 
their teams each week? 

What are your expectations about how this 
meeting time should be used? 

15. To what extent do you expect teachers to 
discuss student learning outcomes? 

(Probe if teachers are expected to look at 
individual, classroom or grade level outcomes, and 
if they are expected to use data to make decisions 
about instruction and interventions.) 

16. Do teachers administer common 
benchmark assessments for 7th grade 
math? 

(Probes: Which assessments? For what 
purposes? What is done with that data? By 
school administrators? By teachers?) 

17. How often do you meet with math 
teachers to discuss student learning 
data and to strategize about modifying 
instruction, curriculum, or programs? 

School community 

18. Please tell me a little bit about your 
community. 


[Probe for a general description of the 
community, such as broad socio-economic 
factors that may affect student learning 
outcomes. For example, if the community is 
growing or declining, if businesses have closed 
recently resulting in increased unemployment, 
etc.] 

19. How would you describe the students 
attending your school? 

[Probe for situations such as having kids from 
different countries who speak different languages, 
having high poverty families, etc.] 

20. How would you assess the level of support 
for education in your community? 
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Appendix B: ASSISTments Study Interview Data 
Representative Quotations from Principal Interviews, by Topic 


Concerns about HW completion rates: 

Q: Do teachers in your school feel that HW 
completion is a problem in general? For 7th gr 
math 

• Minimal concern, or only a small portion of 
the students not doing hw: 

Do we have a small percentage of kids who are 
negligent or let’s just say lazy, unmotivated to 
do it? In some cases the environment at home 
is not conducive and we understand that as 
well. 

...we have not had a significant problem with 
kids getting their math homework in. Both of 
our middle school teachers do not overburden 
kids with so-called “busy work”. Also, they 
allow for class time to work with those few who 
do not get it in on time, or these same kids do 
attend our afternoon study sessions. 

I think the community basically realizes that it’s- 
-I don’t have a lot of parent complaints about 
the amount of homework, the frequency of 
homework, the expectation that it be done. In 
fact, sometimes I have the opposite complaint - 
-my kids don’t seem to have much homework. 

Yes and it’s based upon motivation, based 
upon home support. So I don’t have a number 
about it but depending upon the age group 
especially. 


The teachers have not found that homework 
completion has been a problem. We had a 
problem a few years ago. We implemented 
something at the high school level and also 
at the middle school level. If students are 
consistently not doing homework, teachers will 
do what they call an academic detention. Call 
a parent... 

I wouldn’t say on a broad scale. Students who 
struggle in school- there’s a correlation I would 
say between students who struggle in school 
and not doing homework. And you could 
argue whether it’s chicken and egg. 

I don’t think teachers feel that homework 
completion is an issue. I believe that we have 
processes in place in which kids can come in 
before school to ask questions if they don’t 
understand something. 

There are always some issues as far as getting 
all the kids to, to get all of their homework 
completed. Urn, we have uh a lot of kids that 
are involved in a great deal of after-school 
activities, whether it be athletic or clubs or 
whatever it might be. But budgeting time is 
difficult. We have a homework lab that we offer 
in the mornings for our kids, and this is where it 
takes place and also in the afternoon we have 
one too. 

The majority of students get their homework 
done. We do offer a homework support a 
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couple of nights a week, for students that do 
have difficulty getting it done. 

• More concern, or larger segment of the 
students not doing homework: 

Yes, that’s one of our biggest problems. We 
feel it is lack of motivation for kids to do - to 
complete work, to complete it in a timely 
fashion and to put enough energy and effort 
into it to make it worthwhile.... When a child 
is at the level where they’re missing three 
assignments they stay after school every night, 
five days a week until they’re caught up. 

I would say that I hear it more from the math 
teacher. I’m not sure it’s math specific in terms 
of completion. I haven’t heard so much in 7th 
grade ELA or 7th grade social studies, but 
definitely hear it from 7th grade math. 

Very much so. We were all very concerned. 

Ya, I think that they think that homework 
completion is a problem. And I think specifically 
within math, the inability of students to 
complete homework, it just is a problem. 

Yes. And, you know, what we’ve just 
implemented this year, using some resources, 
our Title I resources, to offer an opportunity for 
students to have an hour of help, if you will, 
with their homework. And it seems that a lot of 
our students are seeking that help for math. 

Probably 25% of our students have a problem 
with homework completion. And when I say 


problem, that it affects their grade. 25, maybe 
even as much as 30, by the time we’re getting 
into 8th grade. 

But for the most part I think, you know, 
probably it’s like 70% regularly [do homework]. 
You know, that’s not a bad turnout of 
homework. 

Yes. Homework completion is an ongoing 
bugaboo. 6th, 7th, and 8th. In in 7th, 
we’ve had teachers who have tried to do the 
weighting, my homework at a zero percent of 
final grade, in the past, as a way to really use, 
you know, performance tasks as the way to 
(?) calculate a grade. And they’ve seen that 
that has a negative effect on the day to day 
homework. So they they’ve certainly found 
that homework completion is an ongoing issue. 

We find that certain sub-groups struggle more, 
uh for example, our [lower] SES population 
struggles. Urn students that, you know, don’t 
have as many risk factors, like attendance 
or, you know, uh low socio-economics or 
behavioral issues, they, you could give them 
loads and loads and loads of homework and 
they would get that done. We (laugh), that’s 
very, it’s a very difficult gap. And so, you know, 
again, that academic focus period is essential 
because that does provide the support that 
students would need. 

We’ve made it a goal of our school to have 
homework completion improve, so we’ve been 
doing some school-wide work around making 
announcements in the morning to encourage 
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students to keep up their good work, and try 
to be very positive that this is the culture and 
this is what’s happening around you, so be 
on board with that. We have the 7th grade 
procedure that we use, where it’s automatic 
stay after school if you don’t have it done. The 
teachers give pretty thorough feedback about 
the homeworks. And I think that has created 
some buy-in around it. 

• Alternative view: questioning the importance 
of homework: 

We’re trying to separate out what students 
know and behavior, and I see homework 
completion as a behavior—you do it or you 
don’t do it, and it might show what you know, 
but it also might not. ... However, the more we 
read, and research supports that homework 
is not the most effective thing for some of the 
reasons I pointed out-family situations, some 
kids go home and let’s face it, they’re taking 
care of a younger sibling, and parents aren’t 
home. There’s a whole - - they’re hungry. 
There’s a whole raft of issues, like some 
students are at a disadvantage, so I think if 
we can take advantage-we have 9 to 5 here 
or whatever, you know, 7 to 2:30, that’s what 
we need to be focusing on and not relying on 
homework but using that as a support is more 
of what we do here to the best that we can. 


Principal’s expectations for how Ts should 
spend meeting time/ work with data: 

Q: What are your expectations about how this 
meeting time should be used? 

Q: To what extent do you expect teachers to 
discuss student learning outcomes? 

Use of team time for just general planning: 

That specific time is supposed to be for-it’s 
the only time that all of them can get together 
on that team. Sometimes they have to deal 
with student issues. But the intent is to really 
use that time to work on these interdisciplinary 
projects and to coordinate with one another... 

Well, I do expect them to really be looking 
at instruction and looking at the curriculum, 
looking at their assessments, making sure 
they’re aligned, making sure they’re making 
them current and, you know, making them 
effective for kids. 

There’s a cycle - they should focus on 
standards, instruction, and progress 
monitoring. They also talk about unit planning, 
assessments, and intervention. They send me 
a weekly update, and I meet monthly with the 
PLC team facilitator. 

Well, they, they should be uh going over current 
curriculum. And obviously currently best practices. 
Uh they should be using the time for uh collegial 
assistance. Collegial assistance of each other. 

Urn and just trying to improve, uh just trying to get 
from each other to improve instruction. 
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Use of team time for examining student 
learning and data: 

We want them to discuss how students are 
doing. We want to know overall like what 
percentage of them are showing growth, 
what percentage is not. What percentage of 
students is in each category. And then the 
students who are the lowest, what are we 
doing to bring them up? (describing use of 
monthly content team mtgs) 

The reason we don’t do the nuts and bolts stuff 
in staff meetings is because that’s where I want 
it done, on the team level. I can disseminate all 
my information by email. I don’t need to talk to 
the crowd. So that’s where they talk about the 
kids. They talk about what they’re doing in the 
classroom because most teams are trying to 
align, you know, what they’re doing with what’s 
going on in the other classrooms. So it’s 
very team centered. In the content meetings, 
they’ll—just that. They’re content meetings and 
they’re more talking about, for example, we 
just did district writing bumps and they were all 
focused on that language. 

Constantly. So we have one meeting a 
month- one meeting a week. It’s called 
Common Planning Time and they are 
supposed to spend that time curriculum and 
assessment, which is hardly enough time, and 
then Wednesday’s are the vertical and the 
team. 

I would assume they are discussing that every 
day.... I walk in on teachers all the time. They 


are sitting down. They are saying “I had so 
and so last year. You have him this year. You 
know I am having trouble with this or that. Can 
you help?” We have RTI meetings on kids and 
develop plans together. 

What we ask teachers to do is at each grade 
level, and as a content area, so math teachers 
in 6th 7th and 8th grade will meet, and they'll 
look at the resources that we have. Try to look 
at say, you know, so what resources do we 
need, in addition to the resources that we have, 
that are going to help us to get to where we 
need to be in May of 2015. 

They should be living it and dreaming it. We’re 
very conscious of our state assessment tests, 
not that we are urn... driven by them, but we 
are certainly guided by the expectations. We 
do want our students to do well. Um we do 
a lot of work in looking at the assessment and 
looking at how our students perform, um from 
the previous year. And then we focus our 
instruction on the areas that students need to 
improve on for the following year. So that, you 
know, that end of the day time, especially when 
we get back our data, the staff work on that 
together. And usually it’s in content groups. 

Those teams today spent today breaking down 
all the math data and the ELA data for NWEA, 
which is our fall testing. So they’ve been able to 
rank all of our students and tell us which, what 
areas of weakness all of those students have. 

So currently they’re working on, um, we’re 
looking at the architecture of how we have RTI 


Principals Discuss Early Implementation of the ASSISTments Online Homework Tutor for Mathematics 
© Copyright 2015 SRI International. 


25 


SRI Education 


throughout our school. ... supporting students 
who are not meeting standards so, you know, 
they generate that goal at the beginning of the 
year. Sometimes it’s based around the math 
goal or the literacy goal or the vocab goal, and 
then they work on it and review their data every 
other week, and plan from that. 

I wish that I could say that they’re all looking 
at data and trying to best meet kids’ needs 
by using data. I think a lot of it, the meeting 
times have to do with planning times, (I: Mhm) 
making sure people are on the same page, 
um moving forward. Possibly talk about some 
individual needs. (I: Mhm) But I think we’re 
trying to make it more focused on data. 

This week my assistant principal and myself 
are meeting with the teams to talk about our 
NWEA data. 

So they’re [content teams] looking at different 
ways they’re uh, what they’re covering for 
content, how they’re doing it, what they’re 
using for strategies. And then at the grade 
level, it’s just uh co-planning to make sure 
that kids across the grade get a common 
expectation.... They’re also looking at common 
assessments and looking at student work 
when they administer common assessments. 

Um uh what I’m really working at, on, with um 
the junior-senior high school, is looking at data. 
Basing their conversations on what they have 
for data. Um and moving forward in that way. 


Use of inservice/ early release days/ PLC 
time to examine data: 

One of the things we started years ago with 
the science and NWEAs and NECAPs and 
that was data analysis because the science 
teacher’s brains work like that. Then I found 
out math teacher’s brains work like that really, 
really good too. So we’ve started doing item 
analysis on both NECAP and NWEAs. And 
looking at where the strengths and weaknesses 
as a whole and then individual students so that 
when we find a student lacking in a skill we can 
address that skill. And we also make plans 
for the school, each of my contents areas they 
discuss the results and they make themselves 
a content goal. It’s our goal to, you know, 
increase the NECAP math scores by two points 
and this is how we’re going to do it. And they 
develop strategies. 

And the other thing that we’re working very 
hard on is our RTI process. We’re fortunate 
that we have early release on Friday’s that we 
devote to PLCs so that that is when we look 
at students and monitor their process and 
evaluate data. 

So it’s a shift we’re making. We’re trying to- 
-first of all it got additional PLC time built into 
our professional development this year which 
is really time totally devoted student outcomes, 
looking at data, sharing best practices, those 
kinds of things. 

So they'll get together as math teachers and 
we also have our vertical meetings. Sometimes 
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the entire 7th grade will get together with the 
entire 6th grade but since they’re not teaching 
common subjects this year our curriculum 
coordinator has set it up so now all the math 
teachers in our building, 5, 6, 7, 8 will get 
together once a quarter and they'll go over this 
data and start saying okay, what are we seeing 
as trends, what are some of the things we need 
to spend more time on and those are the things 
that we’re working on. 

We’ve really tried to make the shift in the last 
5 years to see themselves as professional 
learning communities, you know, without that 
sort of guiding. I guess that sort of guiding 
principle of it has to be about the student work. 
It has to be about the student work as it relates 
to the essential learning targets that have been 
identified. 

Our next district-wide minimum day that we 
have they’re expected to make a presentation 
on the cross-cutting practices that they’ve, 
you know, some of the strategies that they’ve 
implemented in their classroom. 

The PLCs is really for the professional 
development and to continue to um move our 
curriculum ahead. Um, and really that’s a time 
when the departments are looking at NWEA 
data, you know, informal assessments, those 
kinds of things, and seeing what the trends are. 
Where we’re strong, where we’re challenged. 


Principal’s expectations around data: 

They’re expected to look at the data. We’ve 
actually generated these huge—we started 
them, data sheets, on each class last year that 
lists the NWEA scores, their NECAP scores, 
their writing prompt scores and we’ve color 
coded them for ones that are receiving RTI 
services, ones that receive special ed services, 
and then we’ve looked at the kids who are 
falling below the “meets” [proficiency] level and 
trying to figure out how to address their needs. 

I mean we keep a real close eye on which 
students are having difficulties. I think that’s 
one of the strengths here is that we use a lot 
of data to figure out where our kids are not 
meeting the expectations. We have the NWEA 
testing we do that gives us a starting point. 

So we shared that in a faculty meeting and we 
just-and one of their tasks for this last couple 
of months and were going to meet again 
December 6 is to think about how you measure 
student growth. How do you measure whether 
kids are learning in your content area and in 
your class and what we’re going to do is we’re 
going to come up with a list and share with 
each other on how we measure and then were 
going to do a showcase in the spring where 
we are going to have kids demonstrate what 
they’ve learned either demonstrate or perform, 
apply or discuss or present something. 

And that’s really what, you know, what we’re 
asking teachers to examine. So they’re 
looking at student work, they’re looking at the 


Principals Discuss Early Implementation of the ASSISTments Online Homework Tutor for Mathematics 
© Copyright 2015 SRI International. 


27 


SRI Education 


results of student work, they’re looking at a 
breakdown of the results of those students, 
student works, to determine where there may 
be shortcomings in either our curriculum or in 
our instruction. 

That should be driving their instruction, you 
know, we look at them in terms, we do RTI 
regularly, and benchmark assessments, so 
that we know what direction to go in with them 
individually and as a group. 

With customized learning, we’re really focusing 
in on the individual. With the students that are 
in the classroom you might have clusters of 
students at various places. And so you can 
certainly address it that way... 

And the other expectation is that periodically 
they sit at a table and uh look at student work. 
So they learn from each other. 

And the math team I think will have to create 
it as we go. My hope would be that we’re 
pulling down the common assessment data, 
and we’re looking at the literal individual 
performances of students on individual 
problems. And diagnosing where are the 
weaknesses, overall. 

We’ve actually done some of that. Um, but... 
it’s, you have to, uh, we’re working right now 
to get good, good data. So, you know, the, 
we have the NECAP data, and that’s, it’s okay. 
It’s not necessarily 100% beneficial. We have, 
we use the AIMSWEB here. And that too 
gives some quality data, but some of it isn’t 


quite as effective. Uh, so we’re talking about 
how we use formative assessments uh to give 
us data that we can use to inform our teaching 
and better inform where our students are. ... 
For intervention and enrichment. 

Perceptions of Teachers’ interest in using 
ASSISTments / teachers’ expectations related 
to ASSISTments: (treatment sites only) 

Q: From your perspective, how interested are 
the teachers in using ASSISTments? 

Q: What expectations do teachers have 
related to their use of ASSISTments? 

I think the big thing is the time savings that 
they'll get when... and that’s what interested 
me a lot about it was they have a pretty clear 
picture of where kids are solid on the concepts 
or the skills or where they really need to do 
some re-teaching or reinforcing before the 
kids even walk in the door that day, which 
is amazing... .If the homework is corrected 
before you even get there then you jump 
right into the reinforcing. And I think that is 
going to be a great time saver. So that’s their 
expectation is to help them with that part, 
that’s the biggest.... It’s really going to bring to 
light the kids that are really struggling I think, 
who are maybe just the outliers. And that 
becomes an individual student conversation. 
So I think it’s going to be helpful there. And 
then it gives them opportunity to work with 
small groups and really plan out how they’re 
going to differentiate so. 
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He is using the program, right. And very 
motivated. I think it fits into his teaching style 
and, you know, he likes the technology part of 
it and sees the value of having kids use that 
technology. 

I think it allows him to track student’s progress 
better. Better, easier, quicker, once he, you 
know, goes through that learning process. So 
the tracking. He thinks, in the conversations that 
I’ve had with him that it’s - kids are getting it. 

I’m not really sure. They literally just got their 
hands on it. We had some laptop deployment 
issues. We had some book issues. We had to 
order some books that we were short so they 
were a little tardy in getting rolling. 

The two that are piloting it are very excited 
about it and I see that especially in math where 
it’s a pretty linear, you know, process. I think 
they’ll be a lot of people will buy into that. I 
think the two people that are piloting it are 
great-would be great ambassadors for the 

rest of the staff. I spoke with one of them 

at length yesterday about how cool it was 
and, you know, the kids that were-that are 
not necessarily math kids were excited and 
engaged with it when they were going through 
the showcase thing yesterday. 

They went to the training. They see great 
potential in it. I’m not saying they haven’t had 
some frustration because they have but they 
understand that it is a work in progress and 
that they are definitely positive. They were all 
on the webinar last night. 


It really came from her (teacher’s) interest in 
so I think there’s that much more investment. 
Well there’s good investment and I think that 
investment is based on her interest, not being 
told let’s just do it. 

I think he feels that the best part of it is that 
the kids are getting a little bit of help. They 
get- I think he feels that it’s combating so 
many other issues besides just keeping track 
of the homework. It’s kind of flipped the 
classroom around and he feels the structure 
of the way math is being handled this year is 
actually beneficial to the kids. I think he and 
I both feel that we are going to see some 
evidence in our scores this year that will show 
that this help, this assistance that it does bring 
to kids when they’re working on this is getting 
[teacher] more understanding of what the kids 
need literally that day before he even begins. 

I think they expect to have consistency, a clear 
method for tracking students, not waiting for 
progress reports. 

They’re getting far more information about 
students, about all of their students than they 
were ever able to devise on their own. And 
so, you know, they can see they, you know, 
they and it’s not always the same students 

who struggle. So that I think that they are 

excited about what they’re seeing in the math 
portion of their kid’s brain. 

They’re definitely interested and want to want 
to use anything that can help them get those 
students to do homework and succeed. 
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He really thinks it’s helping the kids and uh it, 
you know, because obviously when you have 
a lot of kids and a lot of specific questions, it’s 
just uh it’s just a great support system for them. 

He expects that it will help students make 
quicker progress in learning a skill. By, again, 
not building on bad habits. 

Very interested. They are very interested in 
making it work. They’ve spoken positively 
about it. ...They are hoping that it’s going to 
help them be more effective with using their 
classroom time. I think that they see it as 
providing some useful formative information 
and helping identify areas of concern for 
particular students. Urn, you know, being able 
to be a little bit more targeted. 

.. .both of the teachers are really student 
centered, so I think what they were seeing 
when the kids were buying into it, almost 
instantaneous, then that was a good thing 
for them to see that kids understood the 
importance of it, they were taking it seriously. 
And they were working hard to you know, what 
are the next steps I need to do to you know 
get at the correct answer. So that’s what you 
want. You want kids to be self-motivated. You 
want kids to push themselves. You want them 
to challenge themselves. And then you can 
facilitate that learning. 

She’s you know, was contemplating retiring 
this year coming, which would have been a 
real loss for our school and for our kids. And 
she was skeptical because she has seen 


everything. You know how veteran teachers 
are. But she said, I spoke with her yesterday, 
and she said “(name), this is the best thing. I 
wish I would have had this in my first year of 
teaching.” And what she stressed most with 
her kids was, is the instant feedback for kids. 

She’s very interested. In fact, ironically, we 
were uh presenting to the school board last 
night about different (?) of technology in the 
school currently. And (name) shared about 
ASSISTments, and we’re part of an RSU, and 
urn a couple of board members asked if other 
schools could sign up if it’s a free service. And 
we said well it’s in pilot right now I believe. Urn 
but um but the video that sort of goes along 
with it, that um sort of your video that sort 
of explains a bit about ASSISTments, with 
reference to science and other subject area, I 
think that’s what that caught their attention. So 
I would say that (name) is interested in using it 
and I would say that others as they learn about 
it are very interested in the concept. 

Her general expectations is to, you know, 
increase the homework return, and the 
completion of homework at home. To 
communicate clearly with parents and it just 
holds everybody more accountable I think... 
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Principal’s expectations related to 
ASSISTments: 

What I hope they’ll get is ability to pick and 
choose problems and mathematical queries 
for students to be involved with. And I’m 
hoping that there’s a quick and easy way to 
assess how students are doing, and that that 
can be used in planning instruction, ongoing 
instruction. 

Certainly as a school we’re trying to really 
increase our use of formative assessment and 
that that is a big push. And I think that that 
aspect of this will be really powerful, and that it 
will be to (name)’s advantage to have it... 

I know for us, I mean, certainly it’s student 
learning. You know, we understand with math 
that there are gaps that need to be filled in, 
so that students have a solid foundation to 
continue to build their math knowledge. 

Implementation of ASSISTments (treatment 
sites only) 

Q: How is the implementation of ASSISTments 
going so far in your school? 

• Concern about time to incorporate 
ASSISTments: 

I think implementation is going very well. I think 
the difficulties are just the time demands on 
(name) identifying clues, or hints that they can 
(?) and ... She wants to do the right thing. And 
works hard to really encourage children to have 
a passion for math, as well as understand it. 


So it’s just balancing those time demands, but 
she’s working at it. 

• Delayed launch of ASSISTments: 

I’m not really sure. They literally just got their 
hands on it. We had some laptop deployment 
issues. We had some book issues. We had to 
order some books that we were short so they 
were a little tardy in getting rolling. 

• Students view it as more work: 

.. .the kids will be doing that urn that there'll be 
basically the double entry. They’ll do the work 
in their notebooks and that they will then enter 
it into ASSISTments. I did hear from one of the 
other teachers that that was taking the kids, 
it was taking a little bit of a sales job, to have 
them do that. Because the kids’ experience of 
it was that it was sort of double dipping, you 
know, having to do the work twice. 

• Technology issues/ lack of internet at home: 

...he [teacher] has a few kids who don’t have 
access to computers, but most, you know, 
excuse me, not computers as much as um 
access to the internet. ... they can use it in in 
school, because they have internet here, but 
they just aren’t able to use it when they go 
home. 

...for kids who don’t have computer access 
at home, which are quite a few. So we had 
to accommodate those kids to be able to be 
on the same page and part of the work, which 
I believe- I can’t remember exactly how he 
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solved that but I’m pretty sure kids come in in 
the morning to do their work. 

The biggest barrier was how are we going 
to make sure that each classroom had the 
technology. I mean we didn’t have extra LCD 
projectors hanging around. So I have had to tie 
up, you know, them because they need them in 
their classrooms. We do know that going into 
it. It would have been helpful I think if people 
know ahead of time. 

Just within the last 24 hours, she said (name), 

I made this discovery. Urn, and I think where 
the hint button is, is a problem for the students. 
On their iPads. That they’re hitting the hint 
button, and it’s really not a hint button, uh 
because it marks the problem wrong and gives 
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them the answer. So it’s not giving them a hint. 
It’s giving them the answer and marking what 
they did wrong. 

• Hints not available, just answers: 

...if you go to hint it would give the answer and 
he wanted it to, you know, maybe last a little 
longer. That you get a hint rather than, you 
know, the kid gives up and boom, he’s got the 
answer. 

• Students don’t like the Red X message 

There’s no second chance. Urn, the teachers felt 
that the red X, next to when they get it wrong, is 
such a negative connotation. Uh... you know, like 
the red marking pen, you know, when you when 
you write a paragraph... 
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